These changes uncouple the code for the dbt utility that runs a dbt command from the code that logs dbt commands. This allows for modular use of different loggers and a very flexible ability to customize logging. This addresses the main concerns from the original PR (#841).

Breaking changes

This PR does cause breaking changes with the previous implementation of the dbt utility, but those changes are well worth it for a much cleaner and clearer interface.

Using the new implementation looks like:

import os
from pathlib import Path
from parsons.utilities.dbt import run_dbt_commands, dbtLoggerSlack, dbtLoggerPython

results = run_dbt_commands(
    commands=["dbt run", "dbt test"],
    dbt_project_directory=Path("/path/to/dbt/project"),
    loggers=[dbtLoggerPython, dbtLoggerSlack(webhook=os.environ["SLACK_WEBHOOK"])],
)

The prior implementation was potentially only compatible with Redshift, depending on how the user's dbt profiles.yaml was set up. The new implementation allows the dbt process to inherit the shell environment from the python parent process, and is therefore compatible with running dbt on any database, and the user can configure credential passing using environment variables (recommended) or any other method.

Improved documentation

There is now much more thorough documentation throughout the 4 modules containing the dbt utility code.

TO DO:

• Touch up the dbtLoggerMarkdown text formatting a bit
• dbtLoggerDatabase splits artifacts into a run table and a node table